{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GAN\n",
    "\n",
    "\n",
    "\n",
    "## 1、GAN(Generative Adversarial Networks) 是什么？\n",
    "\n",
    "GAN包含两个网络，一个是generator，一个是discriminator，两个网络相互之间进行对抗，来达到最好的生成效果\n",
    "\n",
    "\n",
    "## 2、GAN的原理是什么？\n",
    "\n",
    "GAN的基本原理其实非常简单，这里以生成图片为例进行说明。\n",
    "\n",
    "假设我们有两个网络，G（Generator）和D（Discriminator）。正如它的名字所暗示的那样，它们的功能分别是：\n",
    "\n",
    "* G是一个生成图片的网络，它接收一个随机的噪声z，通过这个噪声生成图片，记做G(z)。\n",
    "\n",
    "* D是一个判别网络，判别一张图片是不是“真实的”。它的输入参数是x，x代表一张图片，输出D（x）代表x为真实图片的概率，如果为1，就代表100%是真实的图片，而输出为0，就代表不可能是真实的图片。\n",
    "\n",
    "在训练过程中，**生成网络G的目标就是尽量生成真实的图片去欺骗判别网络D。而D的目标就是尽量把G生成的图片和真实的图片分别开来。**\n",
    "\n",
    "这样，G和D构成了一个动态的“博弈过程”。\n",
    "\n",
    "\n",
    "下面这个图就描述了这个过程\n",
    "\n",
    "\n",
    "![GAN](../images/GAN/GAN.jpg)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "## 3、 如何训练GAN呢？\n",
    "\n",
    "我们已经知道了GAN可以看成一个“博弈过程”,它的目标是优化函数：\n",
    "\n",
    "\n",
    "![GAN equal](../images/GAN/GAN_eq.jpg)\n",
    "\n",
    "\n",
    "\n",
    "简单分析一下这个公式：\n",
    "\n",
    "* 整个式子由两项构成。x表示真实图片，z表示输入G网络的噪声，而G(z)表示G网络生成的图片。\n",
    "\n",
    "* D(x)表示D网络判断真实图片是否真实的概率（因为x就是真实的，所以对于D来说，这个值越接近1越好）。而D(G(z))是D网络判断G生成的图片的是否真实的概率。\n",
    "\n",
    "* G的目的：上面提到过，D(G(z))是D网络判断G生成的图片是否真实的概率，G应该希望自己生成的图片“越接近真实越好”。也就是说，G希望D(G(z))尽可能得大，这时V(D, G)会变小。因此我们看到式子的最前面的记号是min_G。\n",
    "\n",
    "* D的目的：D的能力越强，D(x)应该越大，D(G(x))应该越小。这时V(D,G)会变大。因此式子对于D来说是求最大(max_D)\n",
    "\n",
    "\n",
    "上面的公式是论文中的公式，在实际应用中，我们会使用如下公式：\n",
    "\n",
    "\n",
    "**Discriminator 的update**\n",
    "\n",
    "![GAN D](../images/GAN/D.png)\n",
    "\n",
    "\n",
    "\n",
    "**Generator 的update**\n",
    "\n",
    "![GAN G](../images/GAN/G.png)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "以上内容参考 https://zhuanlan.zhihu.com/p/24767059 和 李宏毅老师PPT http://speech.ee.ntu.edu.tw/~tlkagk/courses_MLDS17.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 案例展示：通过GAN来生成手写数字"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "from torchvision import datasets\n",
    "from torchvision import transforms\n",
    "from torchvision.utils import save_image\n",
    "from torch.autograd import Variable\n",
    "from torch.utils.data import DataLoader\n",
    "\n",
    "# 超参数\n",
    "DOWNLOAD = True\n",
    "use_GPU = torch.cuda.is_available()\n",
    "Learning_Rate = 0.0003\n",
    "EPOCH = 200\n",
    "\n",
    "# 图像预处理\n",
    "transform = transforms.Compose([\n",
    "                transforms.ToTensor(),\n",
    "                transforms.Normalize(mean=(0.5, 0.5, 0.5), std=(0.5, 0.5, 0.5))])\n",
    "\n",
    "# 使用 MNIST 数据集\n",
    "mnist = datasets.MNIST(root='./data/', train=True, transform=transform, download=DOWNLOAD)\n",
    "\n",
    "# 包装成 DataLoader\n",
    "data_loader = DataLoader(dataset=mnist, batch_size=100, shuffle=True)\n",
    "\n",
    "# Discriminator Model\n",
    "D = nn.Sequential(\n",
    "    nn.Linear(784, 256),\n",
    "    nn.LeakyReLU(0.2),\n",
    "    nn.Linear(256, 256),\n",
    "    nn.LeakyReLU(0.2),\n",
    "    nn.Linear(256, 1),\n",
    "    nn.Sigmoid())\n",
    "\n",
    "if use_GPU:\n",
    "    D = D.cuda()\n",
    "\n",
    "# Generator Model\n",
    "G = nn.Sequential(\n",
    "    nn.Linear(64, 256),\n",
    "    nn.LeakyReLU(0.2),\n",
    "    nn.Linear(256, 256),\n",
    "    nn.LeakyReLU(0.2),\n",
    "    nn.Linear(256, 784),\n",
    "    nn.Tanh())\n",
    "\n",
    "\n",
    "\n",
    "if use_GPU:\n",
    "    G = G.cuda()\n",
    "\n",
    "\n",
    "# Loss and Optimizer\n",
    "loss_fn = nn.BCELoss()\n",
    "\n",
    "d_optimizer = torch.optim.Adam(D.parameters(), lr=Learning_Rate)\n",
    "g_optimizer = torch.optim.Adam(G.parameters(), lr=Learning_Rate)\n",
    "\n",
    "\n",
    "# Tensor to Variable\n",
    "def to_var(x):\n",
    "    if torch.cuda.is_available():\n",
    "        x = x.cuda()\n",
    "    return Variable(x)\n",
    "\n",
    "# start training\n",
    "for epoch in range(EPOCH):\n",
    "    for i, (images, _) in enumerate(data_loader):\n",
    "        batch_size = images.size(0)\n",
    "        images = to_var(images.view(batch_size, -1))\n",
    "\n",
    "        # 给出正标签和负标签\n",
    "        real_labels = to_var(torch.ones(batch_size))\n",
    "        fake_labels = to_var(torch.zeros(batch_size))\n",
    "\n",
    "        ###             训练 Discriminatro               ###\n",
    "        # BCEloss(x, y) = -y * log(D(x)) - (1-y) * log(1 - D(x))\n",
    "        # 当前 y = 1， 那么我们求得的Loss = -y * log(D(x)) 其中 x 来自于真实数据集\n",
    "        outputs = D(images)\n",
    "        d_loss_real = loss_fn(outputs, real_labels)\n",
    "        real_score = outputs\n",
    "\n",
    "        # 当前 y = 0， 那么我们求得的Loss = -(1-y) * log(1-D(x)) 其中 x 来自生成数据集\n",
    "        z = to_var(torch.randn(batch_size, 64))\n",
    "        fake_images = G(z)\n",
    "        outputs = D(fake_images)\n",
    "        d_loss_fake = loss_fn(outputs, fake_labels)\n",
    "        fake_score = outputs\n",
    "\n",
    "        # 损失计算完成，开始方向传播\n",
    "        d_loss = d_loss_real + d_loss_fake\n",
    "        D.zero_grad()\n",
    "        d_loss.backward()\n",
    "        d_optimizer.step()\n",
    "\n",
    "\n",
    "        ###             训练 Generator               ###\n",
    "        z = to_var(torch.randn(batch_size, 64))\n",
    "        fake_images = G(z)\n",
    "        outputs = D(fake_images)\n",
    "\n",
    "\n",
    "        # 本来我们应该是训练数据 使得 log(D(G(z))) 最大化，但是我们可以采取 minimzing log(1 - D(G(z)))的方式来进行代替\n",
    "        # 这时候，我们将此时的标签看成正例即可。\n",
    "        g_loss = loss_fn(outputs, real_labels)\n",
    "\n",
    "\n",
    "        # 反向传播\n",
    "        D.zero_grad()\n",
    "        G.zero_grad()\n",
    "        g_loss.backward()\n",
    "        g_optimizer.step()\n",
    "\n",
    "\n",
    "        if (i + 1) % 300 == 0:\n",
    "            print('Epoch [%d/%d], Step[%d/%d], d_loss: %.4f, '\n",
    "                  'g_loss: %.4f, D(x): %.2f, D(G(z)): %.2f'\n",
    "                  % (epoch, 200, i + 1, 600, d_loss.data[0], g_loss.data[0],\n",
    "                     real_score.data.mean(), fake_score.data.mean()))\n",
    "\n",
    "    # Save real images\n",
    "    if (epoch + 1) == 1:\n",
    "        images = images.view(images.size(0), 1, 28, 28)\n",
    "        save_image((images.data), './data/test_GAN/real_images.png')\n",
    "\n",
    "    # Save sampled images\n",
    "    fake_images = fake_images.view(fake_images.size(0), 1, 28, 28)\n",
    "    save_image((fake_images.data), './data/test_GAN/fake_images-%d.png' % (epoch + 1))\n",
    "\n",
    "\n",
    "# 保存训练参数\n",
    "torch.save(G.state_dict(), './generator.pkl')\n",
    "torch.save(D.state_dict(), './discriminator.pkl')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练10次的效果图：\n",
    "\n",
    "![GAN equal](../images/GAN/GAN_10.png)\n",
    "\n",
    "训练50次的效果图：\n",
    "\n",
    "![GAN equal](../images/GAN/GAN_50.png)\n",
    "\n",
    "训练100次的效果图：\n",
    "\n",
    "![GAN equal](../images/GAN/GAN_100.png)\n",
    "\n",
    "训练150次的效果图：\n",
    "\n",
    "![GAN equal](../images/GAN/GAN_150.png)\n",
    "\n",
    "训练200次的效果图：\n",
    "\n",
    "![GAN equal](../images/GAN/GAN_200.png)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
